K-Means Clustering based Reinforcement Learning Algorithm for Automatic Control in Robots

Reinforcement learning is key research in automatic control, and hierarchical reinforcement learning is a good solution to the problem of the curse of dimensionality. Hierarchical reinforcement learning can only deal with discrete space, but the state and action spaces in robotic automatic control are continuous. In order to deal with continuous spaces in hierarchical reinforcement learning, we partition the goal into sub-goals according to a modified K-means clustering algorithm. We designed a modified K-means algorithm to discrete continuous state and action spaces, and also proposed corresponding reinforcement learning algorithms on the discrete spaces. Simulation experiments show that, the proposed approach has lower control error and higher reward of Q-function than traditional hierarchical reinforcement learning algorithm.